1. Field of the Invention
The present invention relates to a feature point location determination method and apparatus used for face image recognition and the like.
2. Description of the Related Art
In face recognition using image data and the like, location determination of facial organs or equivalent characteristic parts (hereinafter referred to as feature points) is an important task and often governs recognition performance.
The determined locations of the feature points are used as reference points in normalizing size and rotation of a target image and extracting sub-regions needed for recognition from the target image. To calculate feature values suitable for recognition, it is desired to localize feature points accurately.
X. Song et al [X. Song, C. Lee, G. Xu, and S. Tsuji, “Extracting facial features using partial feature templates and global constraints,” Transactions of the Institute of Electronics, Information and Communications Engineers (D-II), Vol. J77-D-II, No. 8, pp. 1601-1609 (1994)] describe a technique for extracting sets of possible feature points from candidate organs extracted by template matching, comparing the feature points with global geometric constraints modeled in advance, and thereby determining locations of the feature points.
L. Wiskott et al [Wiskott, Laurenz; Fellous, Jean-Marc; Kruger, Norbert; and Von der Malsburg, Christoph; “Face Recognition by Elastic Bunch Graph Matching”, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 19, No. 7, p. 775-779, 1997] describe a technique for searching for location of an organ by placing a candidate location of an organ on grid points which provides geometric constraints and by deforming the grid points such that a feature value on the grid points will coincide with a feature value of an object to be detected. The deformation of the grid points is determined using a predetermined evaluation function based on the agreement between the feature values as well as on amounts of distortion of the grid points.
G. M. Beumer et al [Beumer, G. M.; Tao, Q.; Bazen, A. M.; Veldhuis, R. N. J.; “A landmark paper in face recognition” Automatic Face and Gesture Recognition, 2006. FGR 2006. 7th International Conference on, 10-12 Apr. 2006 Page(s): 6 pp.] describe a technique for correcting locations of feature points using an eigenspace with coordinate values of the feature points used as an input vector.
FIG. 2 is a diagram outlining the technique described by G. M. Beumer et al (2006). In step S21, i.e., in a candidate feature point location determination process, plural identification means identify locations of feature points. FIG. 3 shows an example of locations of feature points to be identified. In this example, fourteen feature points (defined to be a set of feature points) indicated by “X” are determined. For example, feature point 31 corresponds to the outside corner of the eye on the left as viewed in FIG. 3.
In step S22, i.e., in an eigenspace projection process, coordinate values of the feature points determined in the candidate feature point location determination process (step S21) are arranged into an input vector and projected onto an eigenspace. When there are fourteen feature points, input data is a 28-dimensional vector (a vector which has fourteen horizontal coordinate values and fourteen vertical coordinate values as elements). The eigenspace is formed of eigenvectors created from feature point location data by learning in advance. The eigenvectors are created by principal component analysis in advance using plural input vectors (feature point location data) which corresponds to accurate feature point locations. In step S23, a dimension reduction process reduces dimensionality by deleting projected values corresponding to eigenvectors with small eigenvalues. For example, 28-dimensional projection vector has its dimensionality reduced to several dimensions. In step S24, i.e., in an eigenspace inverse projection process, the input vector projected onto the eigenspace is inverse-projected onto a real space using the eigenvector and the projection vector whose dimensionality has been reduced in the dimension reduction process (step S23). In steps S22 to S24, any input vector that has outliers and cannot be expressed in the eigenspace is corrected into a vector which can be expressed in the eigenspace.
That is, in steps S22 to S24, the input vector is corrected based on statistical geometric constraints. FIG. 4 is a diagram illustrating an example in which a feature point location is extracted erroneously in the candidate feature point location determination process (step S21). In this example, feature point 41, which is actually a tip of an eyebrow, has been determined erroneously to be the outside corner of an eye. As an input vector created from the feature point locations shown in FIG. 4 go through steps S22 to S24, the location of feature point 41 is corrected so as to be expressed in the eigenspace. FIG. 5 is a diagram showing an example of a feature point location which results when the input vector is subjected to the eigenspace inverse projection process (step S24). It shows how feature point 41 is corrected into feature point 51. In step S25, a distance calculation process calculates distance between coordinates of each candidate feature point outputted from the candidate feature point location determination process (step S21) and corresponding feature point coordinates corrected in steps S22 to S24. That is, the distance between corresponding feature point locations before and after the correction is calculated. In the example of FIGS. 4 and 5, distance in image coordinates between corresponding feature points 41 and 51 is calculated.
In step S26, a selection process compares the each feature point's distance calculated in the distance calculation process (step S25) with a predetermined threshold and thereby selects either the uncorrected coordinates or the coordinates corrected in steps S22 to S24. The selection process (step S26) selects the corrected coordinates if the calculated distance exceeds the threshold, and selects the uncorrected coordinates if the calculated distance does not exceed the threshold. Reference numeral 27 denotes a group of steps repeated for each feature point and 28 denotes a group of steps repeated for each pair of corresponding feature points. Then, using a set of feature points resulting from the selection process (step S26) as an input vector and repeating a coordinate correction process in steps S22 to S26 multiple times, the technique determines proper locations of feature points.
Techniques, such as the one described by Song et al (1994), which extract multiple sets of feature points as candidates and determine an optimal set of feature points using geometric constraints based on a layout relationship among the feature points have a problem in terms of computational complexity. The technique described by L. Wiskott et al (1997) also involves increased computational complexity because the technique searches for the location of an organ by deforming grid points and checking for the agreement between feature values repeatedly.
The technique described by G. M. Beumer et al (2006) also requires the procedures to be repeated multiple times as described above. Besides, since the technique depends on performance of the candidate feature point location determination process (step S21), even multiple iterations may not provide sufficient accuracy.